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The Education Endowment Foundation (EEF) is an independent grant-making charity dedicated to 
breaking the link between family income and educational achievement, ensuring that children from all 


backgrounds can fulfil their potential and make the most of their talents. 


The EEF aims to raise the attainment of children facing disadvantage by: 


e Identifying promising educational innovations that address the needs of disadvantaged 


children in primary and secondary schools in England; 


e Evaluating these innovations to extend and secure the evidence on what works and can be 


made to work at scale; 


e Encouraging schools, government, charities, and others to apply evidence and adopt 


innovations found to be effective. 


The EEF was established in 2011 by the Sutton Trust, as lead charity in partnership with Impetus 
Trust (now part of Impetus-The Private Equity Foundation) and received a founding £125m grant from 


the Department for Education. 


Together, the EEF and Sutton Trust are the government-designated What Works Centre for improving 


education outcomes for school-aged children. 
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Executive summary 


The project 


Rapid Phonics is a synthetic phonics intervention intended to improve decoding skills and reading 
fluency. It teaches the relationship of word sounds to their corresponding letter groups in a structured 
way. In this evaluation, the intervention was delivered across the transition between primary and 
secondary school to Year 6/7 pupils who had not reached Level 4b in English at the end of Key Stage 
2. 


Rapid Phonics pupils received one-and-a-half hours of tuition per week, in groups of four or fewer, by 
specialist teachers from Norfolk County Council. The intervention was delivered over six weeks in the 
summer term of 2013 (end of primary school) and six weeks in the autumn term of 2013 (start of 
secondary school) to 201 pupils from 22 primary schools and then in 13 secondary schools across 
Norfolk. Pupils were individually randomised to receive the intervention or to a waitlist control group, 
who received the intervention at the end of the trial. 


The project was funded as part of a round of projects dedicated to literacy catch-up for pupils at the 
primary-secondary transition who do not achieve Level 4 in English by the end of Key Stage 2. 


Key conclusions 


1. Rapid Phonics was not found to have a noticeable impact on the primary outcome measure 
of reading comprehension at the end of the intervention. 


2. There was a small improvement in the secondary outcome of decoding, but this did not reach 
statistical significance. 


3. Conducting the intervention during the last term of Year 6 and the first term of Year 7 was not 
the most settled period as there were many disruptions in the school environment and 
conditions may not have been best suited for the children to respond optimally to tutoring. 


4. Continuing the intervention from primary to secondary schools can be logistically 
problematic, with a number of children transferring to schools outside the project or changing 
secondary school soon after arrival. 


5. An area of further research would be to employ the intervention in one continuous period at 
an earlier point in primary school, using a larger sample size and with more focus on children 
receiving free school meals and upon SEN. 


What impact did it have? 


No positive effect size was found for the primary outcome of reading comprehension. A small positive effect 
size was found for the other secondary outcome of non-word reading, but not for the secondary outcome of 
single word reading. None of the measures were found to be statistically significant, suggesting that we do not 
have sufficient evidence to confidently conclude that the observed effect was caused by the programme rather 
than occurring by chance. 


As a result, it is not clear that Rapid Phonics is an effective intervention for improving reading 
comprehension for pupils who have not achieved the expected level of attainment in literacy at the 
end of primary school, when delivered across the primary/secondary transition. 
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How secure is this finding? 


The existing evidence based on phonics training suggests that it is beneficial for young beginner 
readers but may be less so for older readers. Evaluations of a precursor of the Rapid Phonics 
intervention, Sound Discovery, suggest it is an effective intervention for beginner readers when used 
for catch-up in the early stages of formal reading instruction, however these evaluations did not 
include a control group. This is the first independent evaluation of the programme, and the first to use 
a randomised controlled trial methodology. 


The evaluation was set up as an efficacy trial, randomly allocating 201 pupils in 22 primary schools to 
an intervention group or a waitlist control group. The developer led the recruitment and retention of 
the schools, was responsible for the training, and oversaw the provision of the intervention. Efficacy 
trials seek to test interventions in the best possible conditions to see if they hold promise. The minimal 
detectable effect size was relatively large (0.32) which impacts on the overall security of the trial. 


Intention to treat analysis was used (i.e. pupils were compared in the groups to which they were 
originally randomly assigned); blinded invigilation and marking of test papers was undertaken; and 
appropriate analysis (multi-level modelling) was used to enable school effects to be taken into 
account. 


Bias may have been introduced by two secondary schools declining to take part in the intervention. 
The overall attrition rate from the primary outcome was low (11%) and was similar between 
intervention and control groups. The process evaluation suggested that the intervention had been 
delivered with a high level of fidelity to the programme as designed by the developer. 


Overall, this suggests the findings are of moderate security. 


How much does it cost? 


Based on teaching five groups per school, each containing four pupils, the estimated cost of the 
intervention per child is approximately £205, including starter packs for both primary and secondary 
schools and teacher time to deliver the intervention. This per-pupil cost will vary according to the 
number of classes and pupils who require the intervention, and future costs will be less given the 
upfront investment in materials. 


Effect size (95% Estimated 
Coxe) avi ce(=1alers) months’ 
Tals) ave-1 9) progress 


| Coe) j 
pupils 


| =AY (0 (=) a (ors) 
strength* 


Intervention vs 
control (all -0.05 (-0.34, 0.25) -1 month aaa 
pupils) 


Intervention vs 
control (FSM -0.07 (-0.36, 0.23) -1 month 
only) 


*For more information about evidence ratings, see Appendix C in the main evaluation report. Evidence ratings are not provided 
for sub-group analyses, which will always be less secure than overall findings. 
*“For more information about cost ratings, see Appendix D in the main evaluation report. 
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Introduction 


Intervention 


Rapid Phonics teaches the relationship of word sounds to their corresponding letter groups in a 
structured way. Its author is Marlynne Grant (e.g. Grant, 2011). The lessons are interactive and lively, 
incorporating various exercises and word games, and are accompanied by bespoke support materials 
which include books (some in e-book format), posters and worksheets. Although the methodology 
used by Rapid Phonics can be employed in teaching typical beginner readers, in this study Rapid 
Phonics was a catch-up programme tailored to small groups of struggling readers who were aged 
between 10 and 11 at the start of the trial. 


Rapid Phonics is flexible: it can be used as an intervention meeting the needs of older struggling 
readers, including those in Years 6 and 7 (as it is used in this study), but it also conforms to the 
requirements of the Year 1 Phonics Screening Check, and so can be used much earlier in primary 
school. It can be used with groups of up to six children, or for one-to-one teaching when appropriate. 


Background evidence 


Sound Discovery, from which Rapid Phonics is a recent development, was featured as one of the 
leading reading programmes used for struggling readers by Greg Brooks in his publication ‘What 
Works’ (2013). 


There is an existing body of research evidence about the effectiveness of Sound Discovery as a 
reading course for beginner readers and as a catch-up in the early stages of formal reading 
instruction, particularly in connection with following up readers who have fallen behind after Year 1. 
The success of Sound Discovery for beginner readers has been documented by a piece of 
longitudinal action research lasting eight years (Grant, 2011) in which statistical analyses made by the 
local authority compared the progress of the participants vis-a-vis the national picture. It found that 
Sound Discovery teaching received by 700 children (including those from disadvantaged groups) in 
Key Stage 1 led to Key Stage 2 results which were significantly above the national average: for 
example, 65% of pupils gained Level 5 in Key Stage 2 English compared to the national figure of 
27%. No control group was employed in this research. 


Research into Sound Discovery instruction delivered over the same 12-week duration as in the 
current trial was carried out in Norfolk (Worsley, 2005) but without a control group. This was action 
research conducted by the county council which used the Salford sentence reading test (reading 
accuracy) aS an outcome measure, and the participating schools were selected by opportunity 
sampling. Forty-seven children took part. The Sound Discovery instruction resulted in an equivalent 
reading progress of 5.4 to 15.9 months in eleven of the twelve participant schools. 


There was a wide age range of participants, from Year 2 to Year 8, but only two children were in the 
same year groups as studied in the present trial. 


A group of twenty Year 7 students had previously been studied as a piece of action research at a 
secondary school in Gloucestershire: here, the school Special Educational Needs Coordinator 
(SENCo) reported marked advances in their reading ability when Sound Discovery was taught. It was, 
however, a small study, and the course of instruction was substantially longer than that of the present 
intervention as it was taught three times a week for 28 weeks. The participants in the Gloucestershire 
research comprised only the twenty lowest-achieving readers in Year 7; they progressed from reading 
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scores which were below the lowest measurable level to registering reading ages of between 8:05 
and 10:05." 


Rapid Phonics shares its methodology with Sound Discovery, but has additional aspects which are 
particularly attractive for older children. Elements from Sound Discovery which are well suited to a 
reading catch-up scheme have been included in the present intervention—such as lessons dealing 
with advanced code and lessons for harder, polysyllabic words. Sound Discovery materials have thus 
complemented new ones developed for Rapid Phonics in this study. 


Evaluation objectives 


The present randomised controlled trial is an efficacy trial of Rapid Phonics. EEF efficacy trials seek 
to test evaluations in the best possible conditions to see if they hold promise, but do not seek to 
demonstrate that the findings hold at scale in all settings. The project was funded as a series of 
projects dedicated to literacy catch-up for pupils at the primary—secondary transition who do not 
achieve Level 4b in English by the end of Key Stage 2. 


The main question that the trial sought to answer was: Does Rapid Phonics have a noticeable impact 
on the reading comprehension scores of struggling readers in Year 6, as they make the transition to 
Year 7, compared with a randomly selected control group? 


It is the first evaluation of the approach to have a robust control group using a randomised design. 


The process evaluation focused on the fidelity of implementation, so the primary aim here was to 
monitor whether the intervention was being delivered in accordance with the training received by the 
delivery team. It was not the intention of the evaluation to identify areas of improvement in delivery, 
but the interviews and questionnaires with the staff delivering the intervention may include such 
points. Other than these two forms of feedback, lesson observations were the method by which the 
implementation was assessed. 


Project team 


The project manager was Joan McLauchlan (joan.mclauchlan@norfolk.gov.uk). She was responsible 
for recruiting the schools, organising the team’s training in Rapid Phonics, negotiating facilities with 
participating secondary schools, organising specialist teachers’ timetables, purchasing and allocating 
resources, liaising with EEF and the evaluator, arranging for the provision of independent invigilators 
and markers of the tests (who were blinded as to which treatment group the children were part of), 
sending the test results to the evaluator, and conducting the day-to-day administration of the project. 


The project leader was James Thatcher, a Senior Educational Psychologist at Norfolk who was also 
responsible for the grant application (james.thatcher@norfolk.gov.uk). 


The project team comprised a group of specialist teachers from Norfolk County Council. Twelve 
teachers delivered the Year 6 intervention and eight teachers instructed the children in the Year 7 
phase. They were experienced teachers with specialisms in teaching pupils with literacy difficulties 
and advising schools about suitable interventions. They were already well trained and experienced in 
delivering the methodology of the ‘Snappy Lesson’ in Sound Discovery—the lesson format which is 
also featured in Rapid Phonics. A training course for Rapid Phonics attended by these staff, and 
observed by the evaluator, showed that they were well-informed about both phonics and the 
application of that knowledge to the teaching of struggling older readers. The delivery of the course 
was well planned and professional. Discussions revealed that the practitioners were reflective. Ideas 


‘ www.syntheticphonics.net summarises results of this and other Sound Discovery research. 
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were exchanged and suggestions were made as to the best way to deliver the intervention in a way 
that would be faithful to the programme and to the instructions from the author. 


The independent evaluators were from Durham University. Dr Adetayo Kasim was the statistician who 
carried out the analysis of the results and the initial randomisation of the children into the treatment 
groups. The light touch evaluation was conducted by Dr Bernardine King, who liaised with the project 
on matters such as the importance of independent invigilation and marking, the EEF regulations about 
these issues, and the absolute requirement for the testing to be carried out by persons blinded to 
treatment or control allocation. She also ordered the standardised tests, collated the results and wrote 
the final report according to EEF guidance. The report was subject to thorough peer review. 


Ethical review 


The Durham University School of Education Ethics Committee gave ethical approval for this study in 
December 2012. 


EEF guidelines for parental consent were followed and parents were given the opportunity, before the 
start of the trial, to opt their children out from the trial. They were informed that the data would be 
stored by EEF for longitudinal research purposes. The right of the children to withdraw at any time, 
and the preservation of anonymity in reports, were among the points covered in the letters and 
information about the trial that were also shared with parents. Please see the Appendix for the 
parental consent letter. 
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Methodology 


Trial Design 


This was a pragmatic individually randomised trial with two arms, in which the treatment group 
received the intervention and the control group continued their schooling as usual. No control task 
was involved. The intervention group comprised 100 children and the control group consisted of 101 
children. These were drawn from 22 participant primary schools. They went on to attend 19 
secondary schools during the research. The school effect was accounted for in the multi-level 
modelling analysis of results. The trial’s design was chosen to conform to CONSORT standards 
(Altman et al., 2011). 


Randomisation into treatment and control groups meant that the intervention classes did not contain 
the same mixture of children that would have occurred within teaching groups for Rapid Phonics had 
the tuition not been part of a randomised controlled trial. In order to attain a balanced representation 
of reading ability between treatment and control groups, performances in three baseline measures— 
the GL New Group Reading Test (NGRT), the GL Single Word Reading Test (SWRT) and the GL 
Phonological Assessment Battery (PhAB) (in that order)—were included as stratification factors in the 
multi-level modelling. As a consequence, there was less freedom for the project to group together 
children of similar needs than would usually be the case when delivering the intervention. 


Transition studies, when an intervention continues from primary to secondary school, are rare 
(Brooks, 2013). The timing of the trial involved a six-week intervention at the end of primary school 
followed by a six-week summer break and a second six-week intervention at the start of secondary 
school. Individual randomisation was preferred to that of cluster randomisation using schools as the 
unit of randomisation. 


The waiting list condition comprised a ‘business as usual’ group who continued their school activities 
as normal whilst the treatment group received the intervention. Their reading was assessed at the 
same time as their counterparts in the intervention group. At the end of the intervention, these 
comparison children were given the opportunity to receive the intervention by the teachers at their 
schools who had been trained by the Rapid Phonics project team. 


Eligibility 
School inclusion criteria 


Pupil Premium numbers were used by the project manager to determine which schools should be 
approached to take part in the trial. Schools were chosen where the Pupil Premium numbers were 
highest in Norfolk. They were in three areas of the county. 


Pupil inclusion and exclusion criteria 


Children were chosen from the participant schools if they were predicted to achieve a Level 4c or 
below at Key Stage 2 English and if they had a standardised score on the SWRT of less than 100. 
The reason for the latter was that the intervention was intended to improve decoding up to an age- 
appropriate level: those children with standardised scores of 100 or more in the SWRT before the 
start of the study were unlikely to benefit sufficiently to justify missing their usual lessons to attend the 
intervention. 


Decisions about the participation of eligible children with special educational needs were made by the 
schools and the parents before randomisation on the basis of what was appropriate for the individual 
child. 
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Consent from parents was sought before any testing was conducted in each primary school. The 
randomisation was conducted after the pre-tests. 


Intervention 


Table 2: Structure of the intervention instruction 


Stage of Intervention Focus of Training 


Rapid Phonics Step 1 Alphabet sounds 
Rapid Phonics Step 2 Digraph sounds 


Sound Discovery Step 4 Root words, prefixes & suffixes 
Sound Discovery Steps 5-7 Syllables and special endings 


Rapid Phonics Step 3 Alternative spellings 


Table 2 shows the overall structure of the intervention. The earlier stages are especially adapted to 
the Rapid Phonics programme of catch-up tuition whilst the later steps in the intervention are 
supplemented with more advanced materials from Sound Discovery which allow exposure to more 
challenging texts. 


Every week during the six-week treatment periods in both primary and secondary school, the children 
received three half-hour lessons of Rapid Phonics, usually in groups of four or fewer, although with 
occasional one-to-one teaching. All primary schools were offered three lessons of 30 minutes (1.5 
hours) per week: 85% of these schools received these lessons, but due to timetable constraints the 
remaining 15% received two sessions of 45 minutes each week. In secondary school, lesson 
durations were as follows: 


e 70% of schools received three 30-minute lessons a week; 
e 10% of schools received two 60-minute lessons; 
e 10% of schools had two 45-minute lessons; 


e 5% of schools received one 30-minute and one hour-long lesson each week (so 90 minutes 
total); 


e 5% of schools had one 60-minute lesson because it was 1:1 teaching. 


Thus the vast majority of schools received at least the intended 90 minutes of teaching per week. 


The children were taken out of a range of different lessons depending on the decisions made by the 
particular school. The steps at which the children began the intervention were decided by an in-house 
placement test which formed an integral part of the planning the intervention. 


Children were taught by specialist teachers whom they did not know prior to the intervention. Usually 
the children were taught by the same teacher in the secondary school as in primary school, although 
not always. 


During a typical lesson observed by the evaluator, the teacher would recap where they had left off in 
the last lesson and introduce the major teaching point for the lesson—for example letter combinations 
that made the sound of a long ‘u’, such as in ‘blue’, ‘flew and so on. Good use was made of the 
colourful supporting materials, especially the posters which allowed the children to see the progress 
they were making. The ‘Snappy Lesson’ format engaged the children, keeping them occupied and 
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interested as they actively participated in exercises and games intended to explain and consolidate 
the learning points. Phoneme synthesis, and spelling rules such as the split digraph, were explicated 
and practised. Each lesson finished with the teacher reminding the children of a few main points and 
previewing the following lesson. 


A range of exercises make up the ‘Snappy Lesson’ but every lesson varied this content so that the 
children experienced all the exercises or games during the course of the week. This ensured variety. 
Reading fluency was emphasised and reading sentences and short passages aloud was an essential 
part of the lessons. Colourfully illustrated short books of contemporary design were read by the 
children. 


Writing was also part of the lessons. The correct way to form letters and write letter combinations was 
modelled. The children wrote in exercise books and on work sheets when appropriate. A key resource 
used was that of individual magnetic boards with magnetic letters which the children moved around, 
for example to spell out different words containing the same sounds. Through these, they were able to 
show their understanding of spelling patterns, split digraphs and so on.. 


Rapid Phonics is published by Pearson’s and can be purchased by interested parties such as 
teachers, schools or parents. 


The control children carried on with their school lessons as usual but were on a waiting list, receiving 
the intervention at the end of the treatment phase, delivered this time by members of the school staff 
who had received training from the intervention team. 


Outcomes 
Table 3 summarises the three reading outcomes used. 


Table 3: Reading outcome measures 


Outcome Assessment Aspect of Reading Tested by ed ditit-lavme)meetc\erelavel- Ta 
Measure Outcome 


NGRT Comprehension—understanding the Primary 
New Group Reading Test meaning of written sentences and texts. 


PhAB Letters—sounds (grapheme—phoneme) Secondary 
correspondences, phoneme blending, 


Phonological Assessment . é 
mainly regular spellings. 


Battery 


SWRT Regular and irregular word reading without | Secondary 
Single Word Reading Test a context. 


The choice of reading comprehension as the primary outcome was made by EEF as an assessment 
of how well the children would be able to engage with the textual component of the Year 7 curriculum. 
The NGRT reading comprehension test does not assess the skill closest to the expected outcome 
from a phonics intervention, i.e. improved word decoding. Owing to the particular nature of the 
relationship between reading comprehension and decoding, two subsidiary questions were included 
which were tested by two further outcome measures which were if the intervention made a noticeable 
difference to the treatment group compared to controls in terms of: (i) their ability to read single real 
words—both with regular relationships between letters and sounds (graphemes and phonemes) and 
irregular grapheme—phoneme correspondences—and (ii) reading ‘non-words’ (made-up words), 
which was intended to assess reading via the sub-lexical phonological route of grapheme—phoneme 
conversion and phoneme blending (Coltheart, 1978; Coltheart et al., 2001). 
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Primary outcome measure 


The primary outcome measure was the standardised score on the New Group Reading Test (NGRT, 
Burge et al., 2010). As a test of reading comprehension, it was chosen by EEF as the externally valid 
measure reflecting the children’s ability to access the Year 7 curriculum. The paper version of the test 
was chosen for this project because there were concerns around the availability of suitable computer 
facilities in all the 22 primary schools. Additionally, the paper NGRT concentrates more on the 
intended primary outcome of comprehension than does the computerised version which diverts low 
scoring children to phonological tasks after the initial sentence-level comprehension and passage- 
level comprehension is omitted for such children. The paper NGRT is an age-standardised test of 
sentence-level comprehension followed by passage comprehension. It is a parallel form test: 3A was 
used as the pre-test and 3B as the post-test. Raw scores were converted to age-standardised scores 
prior to statistical analysis. 


Secondary outcome measures 


The two secondary measures chosen were selected to assess the attainment of skills normally 
considered as stepping-stones towards improvements in reading comprehension. These skills might 
result from better decoding acquired through phonics training. The test chosen to measure the 
children’s progress in reading at the individual word level was the Single Word Reading Test 
(SWRT6-16, Foster, 2007). The children read the list of single real words out loud to the independent 
tester. There are 60 words in total, ranging from high frequency monosyllabic words such as ‘play’ to 
low frequency words such as ‘colloquial’. The raw test scores were converted to standardised age 
scores for analysis. SWRT has two parallel forms: SWRT1 was administered at pre-test and SWRT2 
at the post-test. So as to monitor any fall-back in scores over the summer holiday, SWRTs were 
administered before and after the summer break to all the children. In summary, all the children, both 
treatment and control groups, were administered the following: SWRT1 in April 2013 (pre-test), 
SWRT2 in July 2013, SWRT1 in September 2013, and SWRT2 in December 2013 (post-test). 


Non-word reading is regarded as a measure of reading via the phonological route (e.g. Colheart et al., 
2001) whereby pseudo-words are read via letter-sound (grapheme—phoneme) conversion, and these 
sounds are then combined (phoneme blending) to form the spoken word. A test of non-word reading 
was taken from the Phonological Assessment Battery (PhAB, Frederickson, Frith & Reason, 1997). 
This test ranges from single-syllable non-words such as ‘pim’ to two-syllable non-words like ‘plutskirl’. 
The child reads the list aloud to the independent tester. The raw test scores were converted to 
standardised scores prior to analysis. 


The invigilators for the primary and secondary measures were unknown to the children, and post-test 
invigilators had no knowledge of which treatment group the children belonged to. They were part of 
the Norfolk specialist teachers’ team. The group testing of the NGRT was carried out under 
examination conditions. 


A random selection of test papers was checked by the evaluator to make sure that they had been 
correctly marked. No mistakes in marking in this selection of papers were noticed by the evaluator. 


Sample size 


The minimum number of participants in the experimental group had been calculated by the evaluators 
as 100, with a further 100 in the control group. These calculations were based on the assumptions 
that (a) the minimum effect size is 0.28, (b) the minimum power value is 0.8, (c) p < 0.05, (d) there is a 
0.7 correlation with covariates, and (e) that half the sample is in the control group. There were no 
readily available data to inform the sample size calculation. The assumed effect size of 0.28 was a 
compromise between resources, recruitment feasibility and the expected impact of the intervention, 
based on existing research (see Background section). 
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The target sample size proposed by the project team was 300, however only 201 children in the 
participating schools were identified by the project as eligible for the study. 


Randomisation 


The goal of randomisation is to ensure that intervention and comparison groups are comparable with 
respect to baseline characteristics. Using a permuted block randomisation scheme, with mixed block 
sizes of three, four, or five,, 101 and 100 children were randomised to the control and intervention 
groups respectively. The randomisation schemes accounted for the baseline scores for NGRT 
standardised scores, PhAB standardised scores, SWRT standardised scores and gender as 
stratification factors. Note that children were randomised across schools. Randomisation within 
schools was not considered because some schools had as little as three pupils in the study. 
Permuted block randomisation with schools and baselines scores as stratification may increase 
accidental imbalance between the intervention groups due to many empty cells. 


The random numbers were generated and the allocation to the treatment and control groups was 
carried out by the independent statistician from Durham University. The randomisation was done 
using Blockrand package in R. The randomisation procedure followed the following steps: 


1. Children were stratified by gender where applicable. 

2. Pupils were further grouped into three performance levels according to NGRT scores; least 
performing pupils, average performing pupils and the high performing pupils. 

3. Within each performance group, pupils were further grouped into block of size three, four or 
five based on a composite score defined per pupil as the best rank from PHAB and SWRT. 
For some pupils the ranking based on the composite score was the same as individual 
ranking by PHAB or SWRT, which may be explained by the correlation between the scores. 

4. Each block of pupils were randomly assigned to either the control or experimental groups 
using a permuted block randomisation scheme. 


The schools which supplied participants for consideration as being eligible for the trial were recruited 
by the project team in Norfolk. The project manager was in charge of informing the schools of the 
children’s allocated groups. The Minimal Detectable Effect Size (MDES) at randomisation for NGRT at 
80% power was estimated as 0.32 based on N = 201 (samples used for analysis) and R* = 0.35. The 
reported MDES was underestimated because it ignored intra-cluster correlation. 


Analysis 


The intervention effect was calculated on the basis of ‘intention to treat’: the data were analysed 
strictly in accordance with the treatment groups to which the children were originally randomly 
allocated. It did not take into account whether the children actually received the intervention, therefore 
in cases where children were swapped from treatment to control or vice versa, their actual treatment 
(or lack of it) would not be taken into account by the intention to treat analysis. Further sensitivity 
analysis was used in situations where children had swapped treatment group. 


The data were analysed using multilevel modelling to account for the clustering of children within 
schools. Although the study was such that randomisation was conducted at the level of the individual 
participant, it is important to account for school effects. If the school effect is removed, then the 
influence of the treatment can be more accurately calculated (in this way, the precise standard error 
for the intervention effects can be derived). Each of the outcomes was analysed separately and the 
results are presented in Table 8 in the Outcomes and Analysis section below. The effect sizes were 
calculated as recommended by Hedges (2007) for clustered data. 


A sub-group of children in receipt of free school meals formed part of the analysis. Appropriate 
statistical analysis of the quantitative outcome measures taking into account pre-test levels was 
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conducted by the statistician at Durham University in order to establish the efficacy of the whole 
intervention. Decoding data from the SWRT was analysed in terms of progress in Year 6, immediately 
before and after the summer holiday, and in Year 7. 


Process evaluation methodology 


Observations of six lessons, at both primary and secondary schools, were carried out. The schools to 
be visited were chosen by the evaluator based on having the opportunity to see a range of schools in 
a variety of neighbourhoods. During these visits, informal interviews took place with delivery staff as 
to their impressions of how the intervention was progressing. The evaluator also attended one training 
session. An email survey of delivery staff was also conducted by the evaluator. Staff were assured 
that their anonymity would be preserved. 
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Impact evaluation 


Timeline 


Date 


November 2012 Project team had recruited three secondary 
schools and their feeder primaries in three 
different parts of the county of Norfolk. 


Pre-testing with NGRT, SWRT and PhAB, 
followed by randomisation in April. 


June-July 2013 Intervention took place in Year 6. 


July 2013 SWRT 2 testing before summer break. 
September 2013 | SWRT 1 testing after summer break. 


September-— Intervention took place in Year 7. 
December 2013 


December 2013 Post-tests of NGRT, SWRT and PhAB. 


Participants 


One hundred children were randomly allocated to the treatment group and 101 to the control group. 


Backgrounds of the schools 


The participant schools were located in the three areas of Norwich, Kings Lynn and Breckland within 
the County of Norfolk. Their catchments ranged from those of marked social deprivation to affluent 
areas. The clusters of schools were chosen because they had high numbers of children eligible for 
Pupil Premium. 


The Income Deprivation Affecting Children Index (IDACI) scores for the areas in which the schools 
were located are shown in Table 4. Twenty-one primary schools were located in urban settings and 
one in a rural setting. Fifteen of the secondary schools were in urban environments and four were in 
rural locations (education.gov.uk). 
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Table 4: IDACI scores for schools participating in the study 


1=¥- Tale komo) | DYN @] crore) cm lave l(er-lalare miavercer-Fy (are me(=) ela ii-14(e) a @ MM anes) me(-) ole hi-\e p) 
0-0.2 0.2-0.25 |0.25-0.35| 0.35-0.4 | 0.4-0.5 | 0.5-0.6 | 0.6-1 


Least Most 
deprived Deprived 


Number | Primary 


of schools 
schools 

falling 

ineach | Secondary 
band of | schools 
IDACI 

scores 


The most recent Ofsted grades awarded to the participating schools varied between 1: ‘Outstanding’ 
and 4: ‘Inadequate’. Table 5 shows the breakdown for the schools. The national background for these 
categories (Ofsted.gov.uk) for state primary schools and primary converter academies, respectively, 
are: 17% and 33% ‘Outstanding’; 61% and 57% ‘Good’; 19% and 10% ‘Requires Improvement’; 2% 
and 0% were ‘Inadequate’. The national background for these categories (Ofsted.gov.uk) for state 
secondary schools and secondary converter academies, respectively, are: 23% and 36% 
‘Outstanding’; 48% and 50% ‘Good’; 24% and 13% ‘Requires Improvement’; 5% and 1% ‘Inadequate’. 


Table 5: Ofsted grades for schools participating in the study 


OTT i r-Tale lave} [eTofere| Requires TakeXe (ete [Ur-1 (cy 
Improvement 


Primary schools | 1 7 11 
(4.5%) (31.8%) (50%) 


Secondary 3 6 4 
schools (15.8%) (31.6%) (31.6%) (21%) 
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Figure 1: Participant flow diagram 


Assessed for eligibility 
(n= 248) Not meeting 


inclusion 
criteria 
(n= 47) 


Allocated to Subverted allocation Allocated to control 
intervention (n= 100 ) (n= 3): were tested and (n= 101) 
analysed according to ITT. 
Did not receive Received control 
intervention and not (n= 100) 
tested: left school 
(n= 2) . Allocation Did not receive 
intervention and not 
Received allocated tested: 
intervention (n=95) left. school: (n= 1). 
Subverted allocation 


Recei allocated (n= 3): were tested and 
intervention (n= 95) analysed according to ITT. 


Lost to follow-up, absent from Lost to follow-up: discontinued 
post-test (n=8). (n= 2) not tested because went 
Discontinued intervention (n=4): Follow-up to secondary schools which did 


went to secondary schools which for primary not opt into trial; 
did not opt into trial outcome (n= 4) no post-test at their 
schools; (n= 2.) absent from 
post-test. 


Analysis of 
Analysed (n= 82) primary Analysed (n= 92) 
outcome 
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Figure 1 is the participant flow diagram for the study, which summarises the loss of participants from 
randomisation to outcome. 


Initially, four main secondary schools and 22 of their feeder primaries agreed to participate. The 
intervention was delivered in all 22 primary schools. Due to children deciding on secondary schools 
outside of the original four, the number of secondary schools attended by participants of the trial was 
eventually 19. Two of these 19 declined to join the study. 


In terms of intention to treat, there was an overall attrition rate of 11.4%, comprising 14% of the 
treatment group and 8.9% of the control group who did not sit the NGRT post-test. Of the total attrition 
of 23 pupils, 13% left for primary schools outside of the area and 26% were lost because they moved 
to secondary schools which were not originally in the trial and which declined to participate. In 
addition, absence on the day of the test was the main cause of attrition, accounting for 60% of the 23 
pupils missing the NGRT post-test. Figure 1 shows that there was some subversion of the random 
allocation whereby three children were accidentally swapped from the treatment to the control group 
and a further three pupils were erroneously swapped from the control group to the treatment group. 
However, these were analysed as intended (intention to treat). This swapping of pupils occurred 
within the same primary school. 


Baseline Characteristics 


The baseline characteristics of the pupils are presented in Table 6. The proportions of the male and 
female pupils were comparable between the intervention and the control groups, which was expected 
due to randomisation. The proportion of children with English as an Additional Language (EAL) was 
also comparable between the intervention and the control groups, however there were minor 
imbalances in the distribution of pupils with Pupil Premium status (PP), those eligible for free school 
meals (FSM), those with Special Educational Needs (SEN), and ethnicity. 


Table 6: Baseline characteristics of the pupils 


Taltcla'ccyalice)a Mere) aline)| Taitca'ccyalice)a Mmm Oxeyalige) | 


“Female 42 (49%) 43 (51%) 78 (50%) 79 (50%) 
Male 58 (51%) 55 (49%) Yes 22 (55%) 18 (45%) 


Pupil Premium (PP) Free School Meals (FSM) 


No A1 (54%) AO (45%) 49 (55%) 
Yes 54 (47%) 60 (53%) Yes 59 (55%) 48 (45%) 
Special Educational Needs (SEN) 
Noo 33 (52%) 31 (48%) 61 (53%) 54 (47%) 
Yes 54 (55%) 22 (51%) 21 (49%) 


The descriptive statistics for the pre-test scores are presented in Table 7. The average standardised 
scores of the outcomes were comparable between the intervention and the control group, which is 
expected due to the randomisation scheme. The spread of the data were also comparable between 
the intervention and the control group. 
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Table 7: Descriptive statistics for the pre-test standardised scores with mean and SD before 
intervention 


Mean (SD) WN CeYolr-Tam(@1 59) After dropout 


NGRT 81.27 (10.07) 81.51 (9.57) 94 (10.00) 95 (12.25) 81.05 (9.91) | 81.97 
(9.31) 


SWRT 82.56 (9.42) 82.92 (8.74) 83.5 (13.00) 84.5 (13.00) | 82.24(9.33) | 83.23 
(8.76) 
PhAB 94.99 (9.36) 95.69 (9.23) 82.5 (12.75) 82.0(11.00) | 94.48 (8.54) | 96.14 
(9.23) 


Note. NGRT = New Group Reading Test; SWRT = Single Word Reading Test; PhAB= Phonological Ability non-word reading 
test; SD = standard deviation; IQR = interquartile range. 


Figures 2 and 3 in the Appendix show the distribution of the NGRT 3A pre-test standardised and raw 
scores. 


Outcomes and analysis 
Intervention effect 


Using intention to treat, the multilevel model showed no evidence of significant intervention effects on 
the primary or secondary outcomes. Although the randomisation was at pupil level, the multilevel 
model accounted for the nested nature of pupils within schools. The model is flexible enough to 
produce the same results as the ordinary ANCOVA model if indeed the heterogeneity between 
schools is zero. Table 8 shows that the effect sizes for the intervention ranged from -0.05 to 0.17, and 
suggests that the intervention was of limited benefit in enhancing reading skills in the children in the 
treatment group compared to the control group, according to intention to treat analysis using age 
standardised scores. Figures 4 and 5 in the Appendix show the distribution of standardised and raw 
scores for the NGRT 3B post-tests. 


Further analyses taking into account other factors such as gender, free school meal status and SEN 
were performed without evidence that these baseline factors had any significant affect. The results of 
the multilevel model with random effects are presented in Table 11 in the Appendix. A separate 
model, using intention to treat, was calculated for FSM pupils (presented in Table 9 below) which 
revealed small effect sizes of 0.2 and 0.3 relating to single word reading and non-word reading 
respectively (intervention compared to control). No improvement was detected in respect of reading 
comprehension (-0.07). None of these measures are statistically significant. 
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Table 8: Intention to treat analysis of the standardised scores with mean and SD after 
intervention—intervention effect (95% Cl) 


Outcome eT Keli] ) | Mean (SD) Effect size (g)* Estimate** 


81.40 (8.94) -0.05 (-0.34, 0.25) -0.35 (-2.53, 1.83) 


een 50 essen | mm 
renin | =8 sears | nome 


Note. *Calculated based on Hedges (2007); **Estimates based on multilevel model to account for school effects; N = number of 
participants; Confidence Intervals (CIs) are shown in brackets in the Effect Size column. Cls represent the possible range of the 
effect size. The MDES was calculated as suggested by Hutchinson and Styles (2010) for cluster randomised trials. 


Table 9: Subgroup analysis for FSM with mean and SD after intervention—intervention effect 
(95% Cl) 


Outcome Group Mean (SD) Effectsize(g)* | Estimate** == MDES | 


Intervention _ 80.13 ICC = 0.00 
NGRT (10.27) 


Control 80.13 -0.07 -0.51 
(8.91) (-0.36, 0.23) (-3.74,2.73) 


Intervention 85.31 iw 0.07 
SWRT (12.06) 
be 


97.56 ICC = 0.03 
(10.61) 


Intervention 
PhAB 


Note. *Calculated based on Hedges (2007); **Estimates based on multilevel model to account for school effects; N = number of 
participants. Confidence Intervals (CIs) are shown in brackets in the Effect Size column: Cls represent the possible range of the 
effect size. The MDES was calculated as suggested by Hutchinson and Styles (2010) for cluster randomised trials. 


2.48 
(0.83,5.99) 


Control 84.17 
(9.66) ae 31, 0.53) 


Sensitivity Analysis 


Table 8 shows the results on an intention to treat (ITT) basis, whereby analysis is conducted strictly 
according to the group to which each participant is allocated, as opposed to whether the participant is 
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actually treated as a member of that particular group. A sensitivity analysis was conducted in addition 
to the ITT analysis. Table 10 shows the results of this sensitivity analysis, where the standardised 
scores of the reading outcomes were analysed using the actual experience of the treatment. This was 
necessary because one of the schools had inadvertently swapped pupils between the intervention 
and control groups. As a result, three children in the treatment group failed to receive the Rapid 
Phonics intervention, whilst three other children who had been allocated to the control group 
mistakenly received the intervention. Table 10 therefore shows the results of the participants 
according to which treatment they actually received, the intervention or control condition. There was a 
small effect size of 0.24 for the PhAB non-word reading scores after adjusting for the pre-test scores. 
This, however, failed to reach statistical significance as the lower confidence interval crossed zero, 
indicating that the null hypothesis cannot be discounted (that there is no significant difference 
between the treatment and control groups). The analyses of the raw scores in Tables 12 and 13 in the 
appendix also show a weak, but not statistically significant, intervention effect on the PhAB scores 
with effect sizes of 0.27 and 0.37 respectively. 


Table 10: Group analysis of the standardised scores based on the actual treatment received, 
with mean and SD after intervention—intervention effect (95% Cl) 


Outcome ej co} tTo) N i (e¥-Talm (21 2)) Effect size (g)* | Estimate** 


e136 (890) | -002(052,027) | 018287, 200) 


eo 
ee 


*Calculated based on Hedges (2007). **Estimates based on multilevel model to account for school effects. 


Missing data 


The data were also investigated for missing data: 6% and 9% of children in the control and 
intervention groups respectively had missing values for PhAB post-test scores; 6% (control) and 8% 
(intervention) had missing values for SWRT post-test scores; and 6% (control) and 12% (Intervention) 
had missing values for NGRT post-test scores. Further analysis based on attendance data was 
limited due to large missing values—95% and 20% for the control and the intervention groups 
respectively. 


SWRT fall-back analysis 


The additional single word tests (SWRT) were intended to assess whether there had been a decline 
in reading scores after the summer holiday. The SWRT did not detect any impact of the intervention, 
either in intention to treat, or in those who actually received the intervention compared to those who 
did not, neither did it find any decline in word reading scores after the summer break. The mean score 
for the intervention group rose from 82.51 in July (SWRT2) to 85.48 (SWRT1) in September, and the 
control score increased from 81.67 to 86.22; these did not constitute significant differences. It was 
concluded, therefore, that there was no fall-back in single word reading after the summer break. See 
Table 14 in the Appendix for the full analysis of SWRT results throughout the intervention. 
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Conclusion 


There was no robust evidence produced by this study to suggest a beneficial effect of Rapid Phonics 
on reading comprehension when delivered across the transition from primary to secondary school. 


Sensitivity analysis comparing those who did and did not experience the intervention revealed that 
there was a small—but statistically non-significant—impact on non-word reading, which is associated 
with decoding using the phonological route. No such impact was detected in the case of single word 
reading recognition (SWRT) and no fall-back after the summer break was detected in single word 
reading. 


Cost 


The publisher of Rapid Phonics supplies a starting pack at a cost of approximately £636, which 
includes teaching guides, a starter pack of readers, online training, an e-book licence, handbook, wall 
charts and flash cards. The individual magnetic boards for the students cost around £5 each and the 
specialised magnetic letter packs (which have, for example, split digraph letter patterns) cost 
approximately £10 each, although these can be shared between pupils, and letters can be 
supplemented by cheaper magnetic letters. 


Teaching costs need to be added to these costs. The average number of students in a teaching group 
for the intervention was four, but as the programme is flexible, schools may decide that bigger 
(probably up to a maximum of six) or smaller groups are appropriate. The estimated cost per school 
to deliver Rapid Phonics to four students over a total period of 18 hours would include approximately 
£540 for an experienced teacher’s time. 


To obtain an estimate of the cost per child, it should be taken into account that this intervention 
involves a primary school and a secondary school each of which would have to purchase the starter 
pack from the publisher and the magnetic boards. Assuming that the intervention is purchased for 20 
children who are taught in five groups of four, the teaching cost of £540 must be multiplied by five—a 
cost of £2,700. This results in an estimated cost per pupil of approximately £205. 


This figure would decline when used for several groups and over subsequent years due to the 
reusability of resources. Specialist training in the teaching of Rapid Phonics is not included in this 
estimate. 


Education Endowment Foundation 22 


Rapid Phonics 


Process evaluation 
A ‘light touch’ process evaluation took place as described above. 


Implementation 


The delivery team made every effort to deliver the intervention as effectively as possible, and were 
able to do this in all but two schools. Children were released from their normal lessons to attend the 
intervention, and the rooms in which lessons took place were acceptable. 


The design of the intervention provided a challenge in that it was in two parts and was implemented at 
a time of disruption for both schools and children. During the last six weeks of primary school there 
are numerous activities such as visits to the children’s secondary schools, school trips and sports 
events which the children do not want to miss: the project staff had to work around these 
commitments. Furthermore, the first six weeks of secondary school is a confusing time for children 
who are in a new environment with very new routines and a different peer group. The project teachers 
sometimes had to go and find students who had forgotten their Rapid Phonics lessons. Unexpected 
situations arose which required a lot of re-allocation of resources in order to overcome. These 
included the children unexpectedly attending nineteen instead of four secondary schools. There were 
substantial logistical problems to overcome in welcoming new schools to the project and in timetabling 
so that staff could teach at the many new locations. If scaled up, this intervention might be best 
timetabled for twelve consecutive weeks in the middle of a school year when the children are all in 
one place, as opposed to moving to different schools. They would be more likely to be settled and in 
an established routine. 


A lot of activity was packed into a short time during the lessons that were observed. The children were 
kept busy. The teaching was responsive to the children’s needs. For instance, when children needed 
more time to absorb a teaching point, the pace of the lesson was adjusted accordingly and more 
examples were given and discussed. In the opinion of the evaluator, the expertise of the teachers 
clearly enhanced the effectiveness of the children’s learning experience. 


Feedback from the delivery team was positive about the Snappy Lesson format and the appropriate 
nature of the support materials for Rapid Phonics. The judgements of the specialist teachers were 
that the intervention was worthwhile, enjoyable for the children, and that it was a programme that 
should be pursued in the future, as its benefit could be seen in the children’s progress. 


Fidelity 


Not all the children received the full number of hours of intervention. Of the 79 children for whom 
accurate records were available, 6 received exactly the intended 18 hours of the intervention, 49 
received more than 18 hours, and 24 received less than 18 hours. The mean number of hours was 
16.1. Absence and the children being called away for end of Year 6 activities were the commonest 
reasons offered for those whose attendance fell below the intended 18 hours. 


Apart from these variations in overall time of delivery and lesson length, the intervention was 
delivered as intended to the treatment group (see Intervention section above). It was the observation 
of the evaluator that the lessons were consistent with the instruction the teachers had received in 
training, and the evaluator’s opinion was that the intervention was professionally delivered to a high 
standard. This is despite some adverse circumstances external to the trial, such as the end of term 
disruptions as described in the Implementation section. The Rapid Phonics programme is very 
adaptable: it can be used to teach a whole class of beginner readers, or may be used for Year 2 
children struggling to keep up with whole-class teaching, or—as in this study—can help older readers 
who are struggling.. 
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Outcomes 


The staff perceived an improvement in the children’s literacy skills and reported an improvement in 
the children’s confidence. No negative effects were reported to the evaluator as to children being 
unhappy at having to attend or about missing lessons. 


Formative findings 


The lesson observations and comments from the teachers delivering Rapid Phonics suggested that 
the content of the intervention is appropriate to meet the need of the children to understand the rules 
that underlie decoding and help them to convert graphemes (letters or letter groupings) to phonemes 
(the sounds in words that correspond to letters / letter groups) and to blend phonemes. The PhAB 
non-word reading results would support this. It is difficult to see obvious improvements in this training 
in reading via the phonological route, which involves grapheme-phoneme conversion and phoneme 
blending. 


In order to improve the skills relating to reading comprehension, the intervention would need to be 
extended to include the teaching of specific sub-skills pertinent to comprehension. As the EEF 
Teaching and Learning Toolkit (Education Endowment Foundation Toolkit, 2014) points out: Phonics 
improves the accuracy of the child’s reading but not the comprehension’. 


Control group activity 


No reports reached the evaluator of the children in the control group displaying any negative 
behaviours. 


There was no evidence of any spill-over effect from the treatment group to the control group. 
However, neither spill over nor placebo effects can be completely discounted. 
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Conclusion 


This pragmatic randomised controlled trial of Rapid Phonics found no positive effect for the primary 
outcome measure of reading comprehension, neither was a beneficial impact observed in the 
secondary outcome of single word reading. Intention to treat and age standardised scores were 
employed in these analyses. Using these same criteria, an effect size of +0.17 was observed for the 
other secondary outcome of non-word reading. This rose to a small positive effect size of +0.24 in the 
sensitivity analysis which took into account the minor allocation subversion (involving six children) and 
was also calculated using standardised scores. When intention to treat groupings and raw scores 
were considered, this rose to +0.27, which represents an estimated benefit for the intervention 
equivalent to three months’ progress in non-word reading. None of these results reached statistical 
significance and so the null hypothesis was accepted—in other words, there was no statistically 
significant difference between the treatment and the business-as-usual control groups after the 
intervention. 


Strengths 


The contribution that can be made by an experienced teacher to the effectiveness of a reading 
intervention is important (Slavin et al., 2011), and the delivery of this intervention was by expert staff. 


This study made use of best practice, conforming to the CONSORT standards for randomised 
controlled trials. Allocation concealment was conducted by the independent statistical evaluator so as 
to prevent selection bias associated with knowledge of trial or control allocation. The intention to treat 
directive ensured that all participants were included in the final analysis according to their original 
allocation. Invigilators and markers were blinded as to the group membership of the participants. 


Limitations 


There were several factors relating to the design of this study, and some unanticipated situations 
occurring during it, which may have reduced the effectiveness of the programme. The last six weeks 
of primary school and the first six weeks of secondary school are times of upheaval in children’s lives 
when they may not be at their most receptive to a period of concentrated study of skills that they have 
always found difficult. The practical problems of providing continuity for the participants going from 22 
primary schools to 19 secondary schools across the county are not to be underestimated, and it is a 
tribute to the management and resourcefulness of the delivery team that its delivery was 
accomplished so well. The original plan targeted four secondary schools and their satellite primary 
schools: when children transferred to secondary schools outside the scheme, it took much persuasion 
to bring these new schools on board. Two secondary schools declined to participate, resulting in four 
children receiving only half of the intervention. 


One possible improvement to the implementation of the intervention would be to schedule it in a 
continuous 12-week stretch (as much as interruptions from half terms would allow) and within the 
same school year. The intervention is likely to have more success with children at an earlier point in 
their primary school careers. Consideration could be given to what complementary skills might be 
taught to Year 6 and Year 7 children to prepare them to access the secondary school curriculum, 
such as vocabulary boosting and the acquisition of higher order cognitive skills. 


The rate of attrition from the research was not high at 11.4%, which is a positive aspect of the study. 
However, only 86 from the treatment group and 92 controls were post-tested for the primary outcome 
of reading comprehension. These figures are considerably below the 150 originally intended for each 
group—treatment and control: if the numbers in each group had been this, or had the number in each 
group reached 100, the results would have been more secure. 
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The SWART failed to find a significant impact on single word reading in the post-tests but neither did it 
detect any decline in this respect following the summer break. There was, however, a small effect size 
suggesting some impact in PhAB non-word reading at post-testing. It would therefore have been 
better to have tested non-word reading before and after the summer break. This was not done 
because PhAB does not exist in parallel forms and assessing the children with the same test on four 
occasions within such a short period was not thought to be valid. As it stands, we cannot make a 
judgement as to whether phonic skills, as shown by non-word reading ability, suffered a post-holiday 
set back. If this timetable for an intervention study were repeated, a test with four parallel forms, such 
as TOWRE2 (Torgesen et al., 2012), should be used. It is also a quick test, assessing real-word and 
non-word reading, and has a timed element which is useful in assessing a programme which 
expressly seeks to improve fluency, as does Rapid Phonics. 


Interpretation 


The main analysis in this randomised controlled trial was conducted on an intention to treat basis. 
Based on the criterion that the primary outcome measure was reading comprehension, there is no 
robust evidence to support the efficacy of Rapid Phonics as an intervention for struggling readers at 
the Year 6/7 transition. This raises the question as to whether a pure phonics scheme can be effective 
in improving reading comprehension for children at the primary—secondary transition without there 
also being an accompanying element of explicit training in comprehension skills. 


No significant improvement was detected in the reading of single words (SWRT). A single word 
reading test provides additional difficulties for readers as words which are ‘tricky’ to decode via 
regular letter-sound conversions are not set within a context that provides clues to assist their 
reading. 


Synthetic phonics programmes aim to teach children both the basic and advanced alphabetic codes 
and to teach the main phonological skills of blending (for reading) and segmenting (for spelling) so 
that when children are reading they do not need to guess from visual cues (such as pictures) or from 
contextual cues. Phonics is their primary strategy for word decoding. But as reading comprehension is 
a product of word decoding and language comprehension children being taught through synthetic 
phonics are provided with opportunities to apply their phonic knowledge and skills from the very 
beginning to real reading and real writing. In other words, their phonics is set within the context of 
language comprehension. 


Marlynne Grant, the author of Rapid Phonics and Sound Discovery, points out that the decodable 
texts within these schemes offer practice with ‘tricky’ words, and that there are also specific advanced 
lessons on this subject, noting that perhaps the short length of the present trial may have limited the 
opportunity for children to master these skills within the time constraints. In connection with reading 
comprehension skills, Dr Grant suggests that phonics training needs time to embed before it 
translates into improved comprehension. 


Another element that should be considered in interpreting the trial’s results is that the reading 
difficulties experienced by the children by the time they reach Year 6 are likely to have hindered their 
acquisition of an age-appropriate reading vocabulary. The lack of measurable improvement in both 
SWRT and NGRT could have been affected by such a deficit. NGRT was not just a challenging test of 
reading vocabulary but also of the general knowledge associated with that vocabulary. The children in 
this study whose decoding ability has improved may improve their comprehension in the future as 
they gain access to the world of books, boost their general knowledge, and develop more holistic 
skills such as understanding text structure and inference-making—key to good reading 
comprehension. 


Achieving a positive result in this randomised controlled trial hinged upon the children’s ability to 
understand some complex texts in the NGRT: such a skill grows with practice and formal training. 
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These children started behind the rest of most of the year group and it is unlikely that the sorts of 
questions asked in the very challenging NGRT would have been accessible to them without such 
practice. For children of this age, improvement in their decoding skills would not be as evident as for 
younger children in comprehension because of the increased complexity and abstractness of the 
language that they encounter in the primary outcome test of this study. 


Future research and publications 


Reading comprehension was intended to test how well this sample of struggling readers could access 
the Year 7 curriculum. The results suggest that they will struggle if they only receive catch-up phonics 
tuition at the point of transition from primary to secondary school. Perhaps earlier intervention would 
be advised and so a recommendation would be to repeat this trial at an earlier point in Key Stage 2. 


This study has highlighted the difficulty of delivering a teaching programme which is split across 
school years and between primary and secondary school. Future research might examine the efficacy 
of the intervention during a more settled period in the children’s school lives, and when it is not 
interrupted by the school summer holiday. 


The improvement in PhAB non-word reading for the treatment group, as evidenced by a small effect 
size (although not statistically significant), suggests that any phonological benefit conferred by the 
intervention had not yet transferred to single word reading or comprehension. Further research could 
examine how phonological improvements brought about by the programme feed into these higher 
order literacy skills. 


The lack of a significant impact in the SWRT could also be investigated. For instance, would 
complementary training in visual holistic word recognition help children of this age group? 


An element of literacy not assessed in this trial was spelling. An intervention study of Sound 
Discovery conducted in a middle school in Bedfordshire between 2005 and 2007 (cited by Brooks, 
2013) reported noticeable progress in spelling among Year 5 pupils. The evaluator observed that 
each lesson incorporated a considerable element of teaching spelling rules. Future research might 
investigate the impact that the Rapid Phonics catch-up programme has on spelling in Years 6 and 7 
within the context of a randomised controlled trial. This would be a logical extension of the research 
objectives of this study as spelling is an essential skill, not least because so much of the assessment 
in secondary school is mediated through the written work the children produce. 


Finally, if these Year 6 and Year 7 struggling readers are empowered through learning phonics to 
decode written material, then the next limiting factor for them must be considered. The difficulty they 
had with the NGRT is consistent with a restriction associated with the unfamiliarity of formal written 
language. This is the type of language that they will become increasingly exposed to as they progress 
at secondary school. Shanahan (2006) suggests that older children are limited in applying their 
decoding skills to their reading comprehension because of the mismatch between their spoken 
vocabulary and the written vocabulary they encounter. To assess this idea, a complementary 
programme to boost written vocabulary knowledge could be investigated in conjunction with Rapid 
Phonics for children before they go to secondary school. 
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Appendix A: Raw scores analysis 


Table 11: Multilevel model results for Table 6—intervention effect (95% Cl) 


eT ce} ty 0) NGRT SWRT PhAB 


Table 12: Intention to treat analysis of the raw scores—intervention effect (95% Cl) 


Outcome ej co}tT) N Mean (SD) Effect size (g)* | Estimate** 


Control 16.41 (6.04) 0.07 (-0.23, 0.36) 0.32 (-1.13, 1.76) 


“conmal—=SSCS StC* g8 81 14 009 (1.9, 180) 
1446 458 0.99 0.11, 1.87 


*Calculated based on Hedges (2007); **Estimates based on multilevel model to account for school effects. 
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Table 13: Group analysis of the raw scores based on the actual treatment received— 
intervention effect (95% Cl) 


Outcome eT colt ye) N iM Cc¥eTa mt (= BD) Effect size (g)* | Estimate** 


NGRT 
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*Calculated based on Hedges (2007); **Estimates based on multilevel model to account for school effects. 


Table 14: Sensitivity analysis of SWRT scores 


Outcome Sroup i Ce¥eTa mt =) BD) Effect size (g)* Estimate** 


emer | [mine 
Se See 
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*Calculated based on Hedges (2007) and adjusted for pre-test scores; **Estimates based on multilevel model to account for 
school effects and pre-test scores. 
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Figure 3 
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Figure 4 
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Figure 5 
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Appendix B: Parental consent letter 


“y Norfolk County Councl 


” at your service 


Snappy catch up phonics: targeted synthetic phonics intervention for children in Norfolk in 
transition from Year 6 to Year 7 


Dear Parents 


In July 2012 the Department for Education welcomed bids from schools and Local Authorities to 
undertake research projects aimed at increasing pupils reading skills in the transition from Primary to 
Secondary schools. 


On behalf of Norfolk County Council, Educational Psychology and Specialist Support submitted a bid 
for funding to undertake research aiming to increase pupils’ reading skills through use of a structured 
phonics intervention. 


Good news was received in November 2012. Our bid was successful. Funding will be provided by the 
Education Endowment Fund (EEF). Pupils in 12 Primary Schools across the county will benefit from 
this project. The children will receive assessment and teaching intervention in Y6 through to their 
transfer into the Autumn term of Y7 to three targeted High Schools. All work will be done in close 
collaboration with school staff. 


Because this is a research project, strict guidelines must be followed to ensure fair and equal access 
to the intervention across the schools involved and to ensure that assessment and teaching is 
undertaken uniformly in every school concerned. The project work will be undertaken by experienced 
and qualified Advisory Learning Support Teachers working closely with teaching staff in schools. In 
many cases the teachers concerned will already be known to the schools. 


In all, around 300 pupils will be assessed to find out if they would benefit from this type of intervention. 
Half of the children (chosen at random) will receive teaching at the end of Yr6 and start of Yr 7 while 
the other half (the control group) will receive it later on during Yr7. 


All of the pupil data will be treated confidentially with children mainly being identified only by their 
Unique Pupil Number and not their names. The E.E.F. may keep the data for study purposes in the 
future. Educational Psychology and Specialist Support will share data with schools in order to support 
the children’s learning in future and names will be used but shared only with the children’s own 
schools. 


All Head Teachers are fully aware and in agreement with the research happening in their schools. If 
you feel that you would prefer your child not to be included in the research please can you tell the 
school. You will be receiving a short letter from them letting you know of the person to contact to do 
this. There will also be a form you can give to the school /f you do not wish your child to take part. It is 
also the case that you can withdraw your child or your child’s data from the project at any time. 
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During the Spring term of 2013, primary schools in the Hewett, Norwich, King Edward VII, Kings Lynn 
and Great Yarmouth Academy, Great Yarmouth clusters will share data with Educational Psychology 
& Specialist Support which identifies pupils who are predicted to gain a National Curriculum Level 2 or 
3 at in the end of Key Stage 2 SATs in May 2013. . 


Of those pupils, 300 will be chosen to have their reading and phonological skills assessed. The 
assessments to be used are fairly quick to administer and will not unduly burden the children. The 
children’s attitudes to literacy will also be rated by a simple scale which they will give information 
towards in conversation with the Advisory Learning Support Teachers because we are also interested 
in boosting their motivation towards literacy on transfer to High School although this is not an integral 
part of the research. 


The project is being independently evaluated by Durham University who will randomly choose around 
150 pupils to have teaching intervention in the summer term of 2013 after the KS 2 SATs are 
completed. 


Teaching will be in small groups of no more than four children. There will be three sessions of around 
45 minutes per week in the six weeks prior to the summer break. The Rapid Phonics teaching 
approach will be used as will access to e-learning reading books. 


In the week just before the children transfer to High School they will meet again with the Advisory 
Learning Support Teachers for a Catch Up session to revise some of the principles of the teaching in 
Y6, share experiences of the e-readers, share reading games and prepare to look forward to sessions 
at High School. 


In the Autumn term of Y7 2013 the same pupils will receive another six weeks of intensive teaching to 
ensure that they remember their learning from the Y6 sessions and to move their reading skills 
forward again. Throughout the project the children’s progress will be regularly assessed in addition to 
the pre and post testing which will evaluate the impact of the intervention. 


Much more detail around the organisation and content of the teaching sessions will take place 
throughout the Spring term and further information will be shared with you through project newsletters 
and parents’ meetings. The general findings of the study will be communicated to parents at the end 
of the project and individual parents can request their own child’s test results. 


It is essential that the children attend all sessions and a record of attendance will be kept. 


We very much hope that this project will be successful in aiding children to improve their reading and 
that you will be happy for your child’s participation. 


Joan McLauchlan, Divisional Senior Specialist Lead Teacher, Project Manager & 
James Thatcher, Divisional Senior Educational Psychologist, Project Lead 
Children’s Services, Educational Psychology & Specialist Support 

Carrow House, Level 2, King Street, Norwich, NR1 2TN 


Telephone 01603 307550 
Email joan.mclauchlan@norfolk.gov.uk, james.thatcher@norfolk.gov.uk 
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Appendix C: Padlock rating 


1.Design: arene 3. Attrition: 4. Balance: see 
What is the ake What is the Adjustment valelity: 
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; start? . : interpretation? 
Evaluation design Implementation Analysis and interpretation 


lat=ilare| 1. Design 2. Power 3. Attrition 4. Balance 5. Threats to 


validity 
Fair and clear experimental 
design (RCT, RDD) 


Well-matched comparison 
(quasi-experiment) 
Matched comparison 
(quasi-experiment) < 40% 
oor or no matchin <U. < © | 
p g Vv V 
No comparator 9 Imbalanced on Pee 


The final security rating for this trial is 3 @. This means that the conclusions have a moderate level of 
security. 


The trial was designed as an efficacy trial and could achieve a maximum of 5 @. This was a well 
conducted trial during the difficult ‘transition’ period. The MDES was relatively large (0.32), which 
resulted in a loss of 2 padlocks. There was moderate attrition (11%), so the padlocks were not 
changed. There was little indication of baseline imbalance or threats to validity. Therefore the final 
padlock rating is 3 @. 
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Appendix D: Cost rating 


Cost ratings are based on the approximate cost per pupil of implementing the intervention over one 
year. Cost ratings are awarded using the following criteria. 


Cost Description 


£ Very low: less than £80 per pupil per year 
££ Low: up to about £170 per pupil per year 


LEE Moderate: up to about £700 per pupil per year 


LEEEL High: up to £1,200 per pupil per year. 


LELEE Very high: over £1,200 per pupil per year 
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You may re-use this document/publication (not including logos) free of charge in any format or 
medium, under the terms of the Open Government Licence v2.0. 


To view this licence, visit www.nationalarchives.gov.uk/doc/open-government-licence/version/2 
or email: psi@nationalarchives.gsi.gov.uk 


Where we have identified any third party copyright information you will need to obtain permission 
from the copyright holders concerned. The views expressed in this report are the authors’ and do not 
necessarily reflect those of the Department for Education. 


This document is available for download at www.educationendowmentfoundation.org.uk 
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